1. Field of the Invention
The present invention relates to an image sensing apparatus and an image sensing method for picking up an image through a process of photoelectrically converting received light into an electric signal.
2. Description of the Related Art
An example of a prior art approach to which the invention is directed is found in Japanese Unexamined Patent Publication No. 2006-50544 which proposes an image sensor configured to perform both linear conversion operation and logarithmic conversion operation. The linear conversion operation is a photoelectric conversion process for integrating electric charges generated by exposure up to a point where a specified amount of light exposure (or the amount of incident light which is hereinafter referred to as a threshold value) is reached and converting a value of integration of the exposure-induced electric charges into a voltage proportional to the value of integration of the exposure-induced electric charges. The logarithmic conversion operation is a photoelectric conversion process for converting the amount of light exposure into a logarithmically compressed voltage without integrating the exposure-induced electric charges after the amount of light exposure has exceeded the aforementioned threshold value.
When this kind of image sensor is used in linear conversion mode, the image sensor produces an output proportional to the amount of electric charge generated by each pixel. Although operation in the linear conversion mode is advantageous in that the image sensor outputs an image signal with high contrast (high gradation performance) even when a subject has low brightness, the image sensor has a narrower dynamic range. On the other hand, operation in logarithmic conversion mode is advantageous in that a wide dynamic range is obtained as the image sensor produces an output derived from the amount of incident light through conversion by using a natural logarithm in this mode of operation. The logarithmic conversion operation however has a drawback in that the output image signal has poor contrast since the image signal is logarithmically compressed.
FIG. 16 is a diagram showing a situation in which a subject sensed by a particular pixel of an image sensor varies from a high-brightness subject to a low-brightness subject during an exposure period as the subject moves in an arrow direction relative to the particular pixel.
If the subject varies from a high-brightness subject to a low-brightness subject having approximately zero brightness during the exposure period as shown in FIG. 16, color fringing occurs in an image obtained by an image sensing apparatus employing this kind of image sensor. This phenomenon of color fringing is explained in detail hereinbelow assuming preconditions (1) to (7) below:
(1) The image sensor is provided with a plurality of color filters which allow light of different wavelength ranges (colors) to pass through and individual pixels of the image sensor have varying sensitivities for the different colors;
(2) For the purpose of the following discussion, two cases are assumed, that is, a case where the aforementioned particular pixel is a pixel provided with a color filter for a first color (hereinafter referred to as the first color pixel) and a case where the aforementioned particular pixel is a pixel provided with a color filter for a second color (hereinafter referred to as the second color pixel);
(3) The first color pixel has a higher sensitivity than the second color pixel;
(4) Exposure time within an exposure period is a relatively long period of time;
(5) Each of “pixel values” hereinafter referred to includes not only a value of a pixel signal output from a pixel but also a signal value corresponding to an electric charge generated by the photoelectric conversion process performed in the pixel;
(6) The high-brightness subject and the low-brightness subject each have uniform brightness throughout an entire area of the subject; and
(7) The high-brightness subject is assumed to take one of two forms, that is, a subject having relatively high brightness among subjects classified as high-brightness subjects (hereinafter referred to as the first high-brightness subject) and a subject having relatively low brightness among the subjects classified as the high-brightness subjects (hereinafter referred to as the second high-brightness subject), wherein the first and second high-brightness subjects are of the same color.
FIG. 17 is a graph showing an example of how the amount of incident light (instantaneous value) varies with time T elapsed from the beginning of exposure (time T=0) of the particular pixel when the subject varies from the high-brightness subject to the low-brightness subject as mentioned above. The example of FIG. 17 indicates that the subject varies from the high-brightness subject to the low-brightness subject at time T1 (T=T1) and exposure is finished at time T2 (T=T2), wherein time T1 is referred to as brightness change timing.
Designated by the symbol A1 in FIG. 17 is a tracing of the instantaneous amount of incident light obtained when the subject is a high-brightness subject classified as the first high-brightness subject. As can be seen from the tracing A1, a constant amount of incident light L1 hits the particular pixel from the beginning of exposure to the brightness change timing T1 at which the amount of incident light L drops approximately to a zero level almost instantaneously, and during a period from the brightness change timing T1 to a point of end-of-exposure timing T2, the amount of incident light L remains approximately to the zero level. Designated by the symbol A2 in FIG. 17 is a tracing of the instantaneous amount of incident light obtained when the subject is a high-brightness subject classified as the second high-brightness subject. It can be seen from FIG. 17 that the tracing A2 shows almost the same pattern as the tracing A1 obtained with the first high-brightness subject except that the amount of incident light L on the particular pixel is L2 which is smaller than L1 (L2<L1) from the beginning of exposure to the brightness change timing T1.
Discussed next with reference to graphs of FIGS. 18 and 19 is how pixel values obtained by the first color pixel and the second color pixel vary with time T. Designated by the symbols G2 and B2 in FIG. 18 are tracings of the pixel values through the lapse of time T obtained by the first and second color pixels, respectively, when the subject is a high-brightness subject classified as a second high-brightness subject. Also, designated by the symbols G3 and B3 in FIG. 19 are tracings of the pixel values through the lapse of time T obtained by the first and second color pixels, respectively, when the subject is a high-brightness subject classified as the first high-brightness subject.
The conventional image sensor discussed here has capability to operate the individual pixels in the aforementioned linear conversion mode and logarithmic conversion mode. Specifically, as depicted in FIGS. 18 and 19, the image sensor causes the individual pixels to perform photoelectric conversion in the linear conversion mode in which each pixel produces a pixel value proportional to an accumulated amount of incident light from the beginning of exposure while the accumulated amount of incident light is smaller than a specific reference value (threshold value) of the accumulated amount of incident light, whereas the image sensor causes the individual pixels to perform photoelectric conversion in the logarithmic conversion mode in which each pixel produces a pixel value obtained by logarithmically compressing the instantaneous amount of incident light when the accumulated amount of incident light becomes equal to or larger than the reference value thereof. Designated by the symbol S1 in FIGS. 18 and 19 is a pixel value of a pixel signal produced by a pixel when the accumulated amount of incident light becomes equal to the reference value thereof. This pixel value is hereinafter referred to as a threshold value S1. In the graphs of FIGS. 18 and 19, a region in which the pixel value is smaller than the threshold value S1 is referred to as a linear conversion region and a region in which the pixel value is equal to or larger than the threshold value S1 is referred to as a logarithmic conversion region.
As depicted in FIG. 18, the pixel value produced by either of the first and second color pixels increases at approximately a constant rate in proportion to the exposure time during a particular time duration within the exposure period while the image sensor picks up an image of the high-brightness subject. Also, since the second color pixel has a higher sensitivity than the first color pixel, the pixel value produced by the first color pixel increases at a higher rate than the pixel value produced by the second color pixel. Accordingly, the pixel values of the first and second color pixels vary as shown by the tracings G2 and B2 in FIG. 18, respectively.
Also, when the high-brightness subject is a second high-brightness subject, the pixel values of both the first and second color pixels increase at relatively low rates so that the brightness change timing T1 is reached before the pixel values exceed the aforementioned threshold value S1. Designated by the symbol S2 in FIG. 18 is the pixel value of the first color pixel and designated by the symbol S3 is the pixel value of the second color pixel at the brightness change timing T1. The amount of incident light on either of the first and second color pixels becomes approximately zero at and beyond the brightness change timing T1 so that the pixel values S2 and S3 of the first and second color pixels remain almost unchanged from the brightness change timing T1 to the end-of-exposure timing T2 (as represented by flat portions of the tracings G2 and B2 in FIG. 18) This means that the particular pixel operates only in the linear conversion region during the exposure period, and the pixel value of the pixel signal output from the particular pixel after the end-of-exposure timing T2 becomes S2 when the particular pixel is a first color pixel whereas the pixel value of the pixel signal output from the particular pixel after the end-of-exposure timing T2 becomes S3 when the particular pixel is a second color pixel.
On the other hand, if the subject is a high-brightness subject classified as a first high-brightness subject, the pixel value produced by either of the first and second color pixels increases at approximately a constant rate in proportion to the exposure time during a particular time duration within the exposure period while the image sensor picks up an image of the high-brightness subject as depicted in FIG. 19. Also, since the second color pixel has a higher sensitivity than the first color pixel, the pixel value produced by the first color pixel increases at a higher rate than the pixel value produced by the second color pixel. Accordingly, the pixel values of the first and second color pixels vary as shown by the tracings G3 and B3 in FIG. 19, respectively.
Also, when the high-brightness subject is a first high-brightness subject, the pixel values of both the first and second color pixels increase at a high rate as compared to a case where the high-brightness subject is a second high-brightness subject as depicted in FIG. 19. However, the brightness change timing T1 is reached before the pixel value of the second color pixel exceeds the aforementioned threshold value S1. Designated by the symbol S3′ in FIG. 19 is the pixel value of the second color pixel at the brightness change timing T1. The amount of incident light on the second color pixel becomes approximately zero at and beyond the brightness change timing T1 so that the pixel value S3′ of the second color pixel remains almost unchanged from the brightness change timing T1 to the end-of-exposure timing T2. This means that the particular pixel operates only in the linear conversion region during the exposure period if the particular pixel is a second color pixel.
In contrast, the pixel value of the first color pixel reaches the threshold value S1 at a point of timing Ts before the brightness change timing T1 is reached. Therefore, the particular pixel acting as the first color pixel performs the logarithmic conversion operation from the timing Ts to the brightness change timing T1, so that the particular pixel (first color pixel) produces a value obtained by logarithmically processing the amount of incident light as the pixel value. As a consequence, the pixel value of the first color pixel instantaneously increases to a value S2′ at the timing Ts and this pixel value S2′ remains unchanged up to the brightness change timing T1 as shown in FIG. 19 since the amount of incident light is constant during this period.
In this case, there arises a problem which is described below. The aforementioned logarithmic conversion region is a region in which the particular pixel produces a pixel value obtained by logarithmically processing the amount of incident light without integrating the amount of incident light or exposure-induced electric charge. Therefore, the pixel value of the first color pixel drops from S2′ at a point of the brightness change timing T1 when the amount of incident light becomes approximately zero. Since the first color pixel holds an accumulated electric charge corresponding to the aforementioned threshold value S1 at this point, the pixel value of the first color pixel drops down to S1 at the timing T1, and this drop in the pixel value occurs instantaneously. Then, the pixel value output from the first color pixel after the point of end-of-exposure timing T2 remains at S1. As the pixel value output from the first color pixel becomes S1, and not S2′, as described above, there occurs color fringing in an image produced by the image sensor.
The ratio (S3′/S3) of the pixel value S3′ of the pixel signal obtained from the second color pixel when the high-brightness subject is a first high-brightness subject to the pixel value S3 of the pixel signal obtained from the second color pixel when the high-brightness subject is a second high-brightness subject coincides with the ratio (L1/L2) of the amount of incident light L1 to the amount of incident light L2 shown in FIG. 17.
On the other hand, the pixel value of the pixel signal expected to be naturally output from the first color pixel upon completion of exposure is a pixel value S2″ shown in FIG. 19 which would be obtained at the point of the brightness change timing T1 if the first color pixel were to perform the linear conversion operation, and not the logarithmic conversion operation, even during a period from the timing Ts to the brightness change timing T1. This pixel value S2″ is the pixel value obtained by the photoelectric conversion process performed during the period from the timing Ts to the brightness change timing T1. The ratio (S2″/S2) of the pixel value S2″ expected to be naturally obtained from the first color pixel when the high-brightness subject is a first high-brightness subject to the pixel value S2 of the pixel signal obtained from the first color pixel when the high-brightness subject is a second high-brightness subject coincides with the ratio (L1/L2) of the amount of incident light L1 to the amount of incident light L2 shown in FIG. 17.
Among the aforementioned pixel values S2, S2″, S3 and S3′, there is a relationship expressed by equation S2″/S2=S3′/S3=L1/L2 from which a relationship expressed by equation S2/S3=S2″/S3′ is derived. If white balance is adjusted based on the ratio between the pixel value of an output signal of the first color pixel and an output signal of the second color pixel, the relationship expressed by S2/S3=S2″/S3′ is obtained so that the high-brightness subject is presented by the same color regardless of whether the high-brightness subject is a first high-brightness subject or a second high-brightness subject.
However, the pixel value of the pixel signal actually obtained from the first color pixel when the high-brightness subject is a first high-brightness subject becomes equal to the threshold value S1 (≠S2″) which does not reflect the result of photoelectric conversion performed from the timing Ts to the brightness change timing T1. Therefore, the ratio S1/S3′ of this pixel value S1 to the pixel value S3′ of the pixel signal obtained from the second color pixel when the high-brightness subject is a first high-brightness subject does not coincide with the ratio S2/S3 of the pixel value S2 to the pixel value S3 which are obtained from the first and second color pixels, respectively, when the high-brightness subject is a second high-brightness subject. It follows that the high-brightness subject is presented in different colors depending on whether the high-brightness subject is a first high-brightness subject or a second high-brightness subject, thereby causing a color fringing problem.